Introduction to Computer Vision: Plant Seedlings Classification

Problem Statement

Context

In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.

Objective

The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.

Data Dictionary

The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.

• The dataset can be download from Olympus.

• The data file names are:

  • images.npy
  • Labels.csv

• Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.

• The goal of the project is to create a classifier capable of determining a plant's species from an image.

List of Species

• Black-grass
• Charlock
• Cleavers
• Common Chickweed
• Common Wheat
• Fat Hen
• Loose Silky-bent
• Maize
• Scentless Mayweed
• Shepherds Purse
• Small-flowered Cranesbill
• Sugar beet

Note: Please use GPU runtime on Google Colab to execute the code faster.

Importing necessary libraries

# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
#!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user

# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.13.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user

Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.

import os
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd                                                                              # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2                                                                                       # Importing openCV for image processing
import seaborn as sns                                                                            # Importing seaborn to plot graphs


# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential                                                   # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix                                                     # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
# Display images using OpenCV
import cv2                                                                                       # Importing cv2
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
from keras import layers
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

Loading the dataset

# Load image file of the dataset
images = np.load("D:/AIML UT AUSTIN/Introduction to Computer Vision/Project5/images.npy")

# Load the labels file of the dataset
labels = pd.read_csv("D:/AIML UT AUSTIN/Introduction to Computer Vision/Project5/Labels.csv")

Data Overview

Understand the shape of the dataset

images.shape

(4750, 128, 128, 3)

• Total 4750 images are there of Shape 128*128 and 3 channel

labels.shape

(4750, 1)

# Create copy of data 
images_bgr = images.copy()

Exploratory Data Analysis

• EDA is an important part of any project involving data.
• It is important to investigate and understand the data better before building a model with it.
• A few questions have been mentioned below which will help you understand the data better.
• A thorough analysis of the data, in addition to the questions mentioned below, should be done.

1. How are these different category plant images different from each other?
2. Is the dataset provided an imbalance? (Check with using bar plots)

def plot_images(images, labels):
  num_classes = 12
  categories = np.unique(labels)
  keys = dict(labels['Label'])
  rows = 5
  cols = 5
  fig = plt.figure(figsize=(12,16))
  for i in range(cols):
    for j in range(rows):
      random_index = np.random.randint(0, len(images))
      ax = fig.add_subplot(rows, cols, i * rows + j + 1)
      ax.imshow(images[random_index, :])
      ax.set_title(keys[random_index])
  plt.show()

plot_images(images_bgr, labels)

Checking data imbalance

order = labels['Label'].value_counts().index
sns.countplot(labels['Label'], order=order)
plt.xticks(rotation=90)
plt.show()

• As we can see data has variations in counts of plant species. But we have sufficients counts to train the model

Data Pre-Processing

Convert the BGR images to RGB images

for i in range(len(images)):
  images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)

# Plot images after BGR to RGB for better understanding
plot_images(images, labels)

Resize the images

# Resize images to 128*128 so as to keep the quality of images and make sure have same size images in data
images_resized=[]
height = 128                   # Complete the code to define the height as 128
width =  128                    # Complete the code to define the width as 128
dimensions = (width, height)
for i in range(len(images)):
  images_resized.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))

plt.imshow(images_resized[30])

<matplotlib.image.AxesImage at 0x1b4544f63d0>

images.shape

(4750, 128, 128, 3)

Apply image processing on the images

• Cconvert image from RGB to HSV

Applymasking for images

• Apply Segmentation
• Apply Sharpeninging

def create_mask_for_plant(image):
    image_hsv = cv2.cvtColor(image, cv2.COLOR_RGB2HSV)

    sensitivity = 35
    lower_hsv = np.array([60 - sensitivity, 100, 50])
    upper_hsv = np.array([60 + sensitivity, 255, 255])

    mask = cv2.inRange(image_hsv, lower_hsv, upper_hsv)
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
    mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
    
    return mask

def segment_plant(image):
    mask = create_mask_for_plant(image)
    output = cv2.bitwise_and(image, image, mask = mask)
    return output

def sharpen_image(image):
    image_blurred = cv2.GaussianBlur(image, (0, 0), 3)
    image_sharp = cv2.addWeighted(image, 1.5, image_blurred, -0.5, 0)
    return image_sharp

# put all those functions together.
def segment(img):

  #image_mask = create_mask_for_plant(img)
  image_segmented = segment_plant(img)
  image_sharpen = sharpen_image(image_segmented)
  return image_sharpen

Itrate on all images

# and `images_resized` is a list of resized images
# Initialize an empty list to store processed images
processed_data_color = []

# Iterate through the images
for indx, image in enumerate(images_resized):
    try:
        # Process the image using the segment function
        processed_image = segment(image)
        
        # Append the processed image to the array
        processed_data_color.append(processed_image)
    
    except Exception as e:
        # Print the index and error details for debugging
        print(f"Error processing image at index {indx}: {e}")
        continue

# Convert the list to a NumPy array for further processing if needed
import numpy as np
processed_data_color = np.array(processed_data_color)

      

processed_data_color.shape

(4750, 128, 128, 3)

plt.imshow(images[2000])

<matplotlib.image.AxesImage at 0x1b4545563d0>

plt.imshow(processed_data_color[2000])

<matplotlib.image.AxesImage at 0x1b42f478550>

Data Preparation for Modeling

• Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
• You'll have to encode categorical features and scale the pixel values.
• You will build a model using the train data and then check its performance

Split the dataset

X_temp, X_test, y_temp, y_test = train_test_split(processed_data_color, labels, test_size=0.1, random_state=42, stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.1, random_state=42, stratify=y_temp)

print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)

(3847, 128, 128, 3) (3847, 1)
(428, 128, 128, 3) (428, 1)
(475, 128, 128, 3) (475, 1)

Encode the target labels

# Convert labels from names to one hot vectors.
lb = LabelBinarizer()
y_train_encoded = lb.fit_transform(y_train)
y_val_encoded=lb.transform(y_val)
y_test_encoded=lb.transform(y_test)

y_train_encoded.shape

(3847, 12)

y_val_encoded.shape

(428, 12)

y_test_encoded.shape

(475, 12)

Data Normalization

# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

Model Building

Model 0 - Base Model with Convolution and Pulling layers

# Clearing Backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# Initializing a sequential model
model = Sequential()

# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))

# Adding max pooling to reduce the size of output of first conv layer
model.add(MaxPooling2D((2, 2)))

# Add two convolution and max-pooling layers activation = relu
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))

model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))

model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model.add(Flatten())

# Adding a fully connected dense layer with 16 neurons
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.3))

# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model.add(Dense(12, activation='softmax'))

# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                      │ (None, 126, 126, 32)        │             896 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D)         │ (None, 63, 63, 32)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_1 (Conv2D)                    │ (None, 61, 61, 64)          │          18,496 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_1 (MaxPooling2D)       │ (None, 30, 30, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_2 (Conv2D)                    │ (None, 28, 28, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_2 (MaxPooling2D)       │ (None, 14, 14, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_3 (Conv2D)                    │ (None, 12, 12, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_3 (MaxPooling2D)       │ (None, 6, 6, 64)            │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 2304)                │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense)                        │ (None, 64)                  │         147,520 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout (Dropout)                    │ (None, 64)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense)                      │ (None, 12)                  │             780 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 241,548 (943.55 KB)

 Trainable params: 241,548 (943.55 KB)

 Non-trainable params: 0 (0.00 B)

Fitting the model on training data

history = model.fit(
    train_datagen1.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
    epochs=40,
    validation_data=(X_val_normalized, y_val_encoded),
    batch_size=32,
    verbose=2)

Epoch 1/40
121/121 - 13s - 106ms/step - accuracy: 0.2880 - loss: 2.1164 - val_accuracy: 0.4252 - val_loss: 1.6812
Epoch 2/40
121/121 - 11s - 94ms/step - accuracy: 0.4268 - loss: 1.6503 - val_accuracy: 0.4836 - val_loss: 1.4830
Epoch 3/40
121/121 - 12s - 97ms/step - accuracy: 0.4931 - loss: 1.4285 - val_accuracy: 0.5304 - val_loss: 1.3171
Epoch 4/40
121/121 - 12s - 97ms/step - accuracy: 0.5529 - loss: 1.2550 - val_accuracy: 0.5701 - val_loss: 1.2039
Epoch 5/40
121/121 - 12s - 99ms/step - accuracy: 0.5851 - loss: 1.1478 - val_accuracy: 0.6192 - val_loss: 1.0820
Epoch 6/40
121/121 - 12s - 98ms/step - accuracy: 0.6296 - loss: 1.0271 - val_accuracy: 0.7126 - val_loss: 0.8760
Epoch 7/40
121/121 - 11s - 94ms/step - accuracy: 0.6652 - loss: 0.9314 - val_accuracy: 0.7407 - val_loss: 0.7630
Epoch 8/40
121/121 - 11s - 93ms/step - accuracy: 0.7091 - loss: 0.8156 - val_accuracy: 0.6939 - val_loss: 0.8779
Epoch 9/40
121/121 - 11s - 92ms/step - accuracy: 0.7408 - loss: 0.7504 - val_accuracy: 0.7593 - val_loss: 0.7003
Epoch 10/40
121/121 - 11s - 91ms/step - accuracy: 0.7663 - loss: 0.6693 - val_accuracy: 0.7991 - val_loss: 0.5737
Epoch 11/40
121/121 - 11s - 94ms/step - accuracy: 0.7835 - loss: 0.6137 - val_accuracy: 0.8201 - val_loss: 0.5372
Epoch 12/40
121/121 - 11s - 90ms/step - accuracy: 0.7936 - loss: 0.5965 - val_accuracy: 0.8318 - val_loss: 0.5336
Epoch 13/40
121/121 - 11s - 91ms/step - accuracy: 0.8079 - loss: 0.5629 - val_accuracy: 0.7944 - val_loss: 0.6155
Epoch 14/40
121/121 - 11s - 91ms/step - accuracy: 0.8139 - loss: 0.5184 - val_accuracy: 0.8435 - val_loss: 0.4824
Epoch 15/40
121/121 - 11s - 91ms/step - accuracy: 0.8154 - loss: 0.5156 - val_accuracy: 0.8645 - val_loss: 0.4784
Epoch 16/40
121/121 - 11s - 90ms/step - accuracy: 0.8180 - loss: 0.5092 - val_accuracy: 0.8598 - val_loss: 0.4528
Epoch 17/40
121/121 - 11s - 90ms/step - accuracy: 0.8352 - loss: 0.4700 - val_accuracy: 0.8435 - val_loss: 0.4927
Epoch 18/40
121/121 - 11s - 90ms/step - accuracy: 0.8326 - loss: 0.4749 - val_accuracy: 0.8575 - val_loss: 0.4768
Epoch 19/40
121/121 - 11s - 90ms/step - accuracy: 0.8417 - loss: 0.4497 - val_accuracy: 0.8481 - val_loss: 0.4723
Epoch 20/40
121/121 - 11s - 91ms/step - accuracy: 0.8500 - loss: 0.4228 - val_accuracy: 0.8341 - val_loss: 0.4691
Epoch 21/40
121/121 - 11s - 91ms/step - accuracy: 0.8529 - loss: 0.4349 - val_accuracy: 0.8598 - val_loss: 0.4427
Epoch 22/40
121/121 - 11s - 91ms/step - accuracy: 0.8477 - loss: 0.4274 - val_accuracy: 0.8341 - val_loss: 0.4583
Epoch 23/40
121/121 - 11s - 90ms/step - accuracy: 0.8570 - loss: 0.4013 - val_accuracy: 0.8715 - val_loss: 0.4059
Epoch 24/40
121/121 - 11s - 89ms/step - accuracy: 0.8687 - loss: 0.3739 - val_accuracy: 0.8762 - val_loss: 0.3943
Epoch 25/40
121/121 - 11s - 90ms/step - accuracy: 0.8602 - loss: 0.3775 - val_accuracy: 0.8575 - val_loss: 0.4226
Epoch 26/40
121/121 - 11s - 90ms/step - accuracy: 0.8591 - loss: 0.3857 - val_accuracy: 0.8668 - val_loss: 0.3998
Epoch 27/40
121/121 - 11s - 92ms/step - accuracy: 0.8695 - loss: 0.3724 - val_accuracy: 0.8668 - val_loss: 0.3896
Epoch 28/40
121/121 - 11s - 90ms/step - accuracy: 0.8726 - loss: 0.3514 - val_accuracy: 0.8692 - val_loss: 0.3669
Epoch 29/40
121/121 - 11s - 90ms/step - accuracy: 0.8734 - loss: 0.3422 - val_accuracy: 0.8575 - val_loss: 0.4220
Epoch 30/40
121/121 - 11s - 92ms/step - accuracy: 0.8674 - loss: 0.3653 - val_accuracy: 0.8668 - val_loss: 0.4130
Epoch 31/40
121/121 - 11s - 91ms/step - accuracy: 0.8789 - loss: 0.3444 - val_accuracy: 0.8692 - val_loss: 0.4274
Epoch 32/40
121/121 - 11s - 93ms/step - accuracy: 0.8807 - loss: 0.3299 - val_accuracy: 0.8762 - val_loss: 0.3678
Epoch 33/40
121/121 - 12s - 95ms/step - accuracy: 0.8838 - loss: 0.3126 - val_accuracy: 0.8738 - val_loss: 0.3993
Epoch 34/40
121/121 - 11s - 94ms/step - accuracy: 0.8822 - loss: 0.3108 - val_accuracy: 0.8621 - val_loss: 0.4016
Epoch 35/40
121/121 - 11s - 94ms/step - accuracy: 0.8885 - loss: 0.2952 - val_accuracy: 0.8668 - val_loss: 0.3738
Epoch 36/40
121/121 - 11s - 93ms/step - accuracy: 0.8835 - loss: 0.3049 - val_accuracy: 0.8715 - val_loss: 0.3782
Epoch 37/40
121/121 - 11s - 92ms/step - accuracy: 0.8926 - loss: 0.2863 - val_accuracy: 0.8808 - val_loss: 0.3587
Epoch 38/40
121/121 - 11s - 94ms/step - accuracy: 0.8913 - loss: 0.2821 - val_accuracy: 0.8668 - val_loss: 0.4355
Epoch 39/40
121/121 - 11s - 92ms/step - accuracy: 0.8804 - loss: 0.3137 - val_accuracy: 0.8832 - val_loss: 0.4095
Epoch 40/40
121/121 - 11s - 94ms/step - accuracy: 0.8958 - loss: 0.2745 - val_accuracy: 0.8902 - val_loss: 0.3714

Plot Accuracy graph

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Evaluate the model on Test Data

test_accuracy0 = model.evaluate(X_test_normalized, y_test_encoded, verbose=1)

15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - accuracy: 0.8846 - loss: 0.4190

• Model is giving around 89% accuracy on Training and Validation set and around 88% on test set
• Model is generalizing well on data

Generating Predictions on Test Data

y_pred = model.predict(X_test_normalized)

15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 23ms/step

Plot Confusion Matrix

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)

# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()

# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))

                           precision    recall  f1-score   support

              Black-grass       0.37      0.27      0.31        26
                 Charlock       0.95      0.90      0.92        39
                 Cleavers       0.92      0.83      0.87        29
         Common Chickweed       0.94      0.98      0.96        61
             Common wheat       0.78      0.82      0.80        22
                  Fat Hen       0.96      0.94      0.95        48
         Loose Silky-bent       0.75      0.89      0.82        65
                    Maize       0.88      1.00      0.94        22
        Scentless Mayweed       0.94      0.94      0.94        52
          Shepherds Purse       1.00      0.74      0.85        23
Small-flowered Cranesbill       0.96      0.96      0.96        50
               Sugar beet       0.92      0.92      0.92        38

                 accuracy                           0.88       475
                macro avg       0.86      0.85      0.85       475
             weighted avg       0.88      0.88      0.88       475

Comments

• Recall is very low for other classes recall is good

Precision is low for Black grass but all other classes have good preicision and f1 score* Even from confusion matrix, we see model did not perform well for black gra, it is misclassifying black grass as loose silky bent

• Let's see if we can improve performance furtherher

Model Performance Improvement

Model 1 - Model with Data Augmentation

Data Augmentation

Remember, data augmentation should not be used in the validation/test data set.

• As we can see, our initial model appears to be not fitting well. Therefore we'll try to address this problem with data augmentation to check if we can improve the model's performance.

# set the rotation_range to 20
train_datagen1 = ImageDataGenerator(rotation_range=20,  # randomly rotate images in the range
        horizontal_flip=True,  # randomly flip images horizontally
        vertical_flip=True  # randomly flip images vertically
    )

Let's clear the previous model history and set the seed for random number generators in Numpy, the Random library in Python, and in TensorFlow to be able to reproduce the same results every time we run the code.

# Clearing Backend
from tensorflow.keras import backend
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# Initializing a sequential model
model1 = Sequential()

# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model1.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))

# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2)))

# Add two convolution and max-pooling layers activation = relu
model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))

model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))

model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))

model1.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model1.add(MaxPooling2D((2, 2)))

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())

# Adding a fully connected dense layer with 16 neurons
model1.add(Dense(64, activation='relu'))
model1.add(Dropout(0.4))

# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))

# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model1.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                      │ (None, 126, 126, 32)        │             896 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D)         │ (None, 63, 63, 32)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_1 (Conv2D)                    │ (None, 61, 61, 64)          │          18,496 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_1 (MaxPooling2D)       │ (None, 30, 30, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_2 (Conv2D)                    │ (None, 28, 28, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_2 (MaxPooling2D)       │ (None, 14, 14, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_3 (Conv2D)                    │ (None, 12, 12, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_3 (MaxPooling2D)       │ (None, 6, 6, 64)            │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_4 (Conv2D)                    │ (None, 4, 4, 64)            │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_4 (MaxPooling2D)       │ (None, 2, 2, 64)            │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 256)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense)                        │ (None, 64)                  │          16,448 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout (Dropout)                    │ (None, 64)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense)                      │ (None, 12)                  │             780 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 147,404 (575.80 KB)

 Trainable params: 147,404 (575.80 KB)

 Non-trainable params: 0 (0.00 B)

callback_es = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=20, min_delta=0.0001, restore_best_weights=True)

Fitting the model on train data

history = model1.fit(
    train_datagen1.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
    epochs=60,
    validation_data=(X_val_normalized, y_val_encoded),
    batch_size=32,
    verbose=1,
    callbacks=[callback_es])

Epoch 1/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 13s 92ms/step - accuracy: 0.1673 - loss: 2.3368 - val_accuracy: 0.3224 - val_loss: 1.9645
Epoch 2/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.3390 - loss: 1.8291 - val_accuracy: 0.4042 - val_loss: 1.6134
Epoch 3/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.4112 - loss: 1.6411 - val_accuracy: 0.4439 - val_loss: 1.5044
Epoch 4/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.4714 - loss: 1.4943 - val_accuracy: 0.4930 - val_loss: 1.4253
Epoch 5/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 12s 95ms/step - accuracy: 0.5055 - loss: 1.3512 - val_accuracy: 0.6262 - val_loss: 1.0779
Epoch 6/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 94ms/step - accuracy: 0.5750 - loss: 1.1891 - val_accuracy: 0.6612 - val_loss: 1.0131
Epoch 7/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6146 - loss: 1.0839 - val_accuracy: 0.6285 - val_loss: 1.0328
Epoch 8/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.6435 - loss: 1.0078 - val_accuracy: 0.7126 - val_loss: 0.8234
Epoch 9/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6607 - loss: 0.9300 - val_accuracy: 0.6939 - val_loss: 0.8777
Epoch 10/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6730 - loss: 0.9037 - val_accuracy: 0.6706 - val_loss: 0.9063
Epoch 11/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.6966 - loss: 0.8713 - val_accuracy: 0.7523 - val_loss: 0.7496
Epoch 12/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.7128 - loss: 0.8162 - val_accuracy: 0.7664 - val_loss: 0.6761
Epoch 13/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.7374 - loss: 0.7429 - val_accuracy: 0.7827 - val_loss: 0.6893
Epoch 14/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.7486 - loss: 0.7266 - val_accuracy: 0.7921 - val_loss: 0.6222
Epoch 15/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.7676 - loss: 0.6264 - val_accuracy: 0.8318 - val_loss: 0.5579
Epoch 16/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.7933 - loss: 0.5990 - val_accuracy: 0.8201 - val_loss: 0.5970
Epoch 17/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8117 - loss: 0.5618 - val_accuracy: 0.8248 - val_loss: 0.5239
Epoch 18/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8129 - loss: 0.5424 - val_accuracy: 0.8341 - val_loss: 0.5148
Epoch 19/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8047 - loss: 0.5400 - val_accuracy: 0.8224 - val_loss: 0.5354
Epoch 20/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8124 - loss: 0.5347 - val_accuracy: 0.8458 - val_loss: 0.5002
Epoch 21/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8260 - loss: 0.5116 - val_accuracy: 0.8341 - val_loss: 0.5318
Epoch 22/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8255 - loss: 0.5059 - val_accuracy: 0.7991 - val_loss: 0.6077
Epoch 23/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8203 - loss: 0.5013 - val_accuracy: 0.8294 - val_loss: 0.5315
Epoch 24/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8296 - loss: 0.4822 - val_accuracy: 0.8762 - val_loss: 0.4368
Epoch 25/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8457 - loss: 0.4307 - val_accuracy: 0.8598 - val_loss: 0.4601
Epoch 26/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8547 - loss: 0.4229 - val_accuracy: 0.8692 - val_loss: 0.4334
Epoch 27/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8419 - loss: 0.4208 - val_accuracy: 0.8715 - val_loss: 0.4182
Epoch 28/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8581 - loss: 0.4021 - val_accuracy: 0.8598 - val_loss: 0.4848
Epoch 29/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8547 - loss: 0.4118 - val_accuracy: 0.8248 - val_loss: 0.5490
Epoch 30/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.8572 - loss: 0.3857 - val_accuracy: 0.8855 - val_loss: 0.4121
Epoch 31/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8570 - loss: 0.4008 - val_accuracy: 0.8808 - val_loss: 0.3892
Epoch 32/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8643 - loss: 0.3591 - val_accuracy: 0.8621 - val_loss: 0.4287
Epoch 33/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8674 - loss: 0.3665 - val_accuracy: 0.8738 - val_loss: 0.4338
Epoch 34/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8594 - loss: 0.3749 - val_accuracy: 0.8692 - val_loss: 0.4538
Epoch 35/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8846 - loss: 0.3269 - val_accuracy: 0.8808 - val_loss: 0.4001
Epoch 36/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8781 - loss: 0.3417 - val_accuracy: 0.8738 - val_loss: 0.4083
Epoch 37/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8659 - loss: 0.3473 - val_accuracy: 0.8738 - val_loss: 0.4490
Epoch 38/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8758 - loss: 0.3322 - val_accuracy: 0.8785 - val_loss: 0.3907
Epoch 39/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.8749 - loss: 0.3317 - val_accuracy: 0.8832 - val_loss: 0.3886
Epoch 40/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8762 - loss: 0.3302 - val_accuracy: 0.8738 - val_loss: 0.4124
Epoch 41/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8775 - loss: 0.3302 - val_accuracy: 0.8855 - val_loss: 0.3810
Epoch 42/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8828 - loss: 0.3158 - val_accuracy: 0.8855 - val_loss: 0.3606
Epoch 43/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8781 - loss: 0.3289 - val_accuracy: 0.8785 - val_loss: 0.3780
Epoch 44/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8869 - loss: 0.3027 - val_accuracy: 0.8692 - val_loss: 0.3717
Epoch 45/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 90ms/step - accuracy: 0.9007 - loss: 0.2668 - val_accuracy: 0.8925 - val_loss: 0.3632
Epoch 46/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8876 - loss: 0.2920 - val_accuracy: 0.8575 - val_loss: 0.4948
Epoch 47/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8729 - loss: 0.3530 - val_accuracy: 0.8785 - val_loss: 0.3838
Epoch 48/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8909 - loss: 0.2890 - val_accuracy: 0.8598 - val_loss: 0.4077
Epoch 49/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8799 - loss: 0.3279 - val_accuracy: 0.8692 - val_loss: 0.4255
Epoch 50/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8905 - loss: 0.3029 - val_accuracy: 0.8879 - val_loss: 0.3923
Epoch 51/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9045 - loss: 0.2473 - val_accuracy: 0.8925 - val_loss: 0.3635
Epoch 52/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8908 - loss: 0.2875 - val_accuracy: 0.8785 - val_loss: 0.4018
Epoch 53/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.9037 - loss: 0.2575 - val_accuracy: 0.8879 - val_loss: 0.4274
Epoch 54/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.8757 - loss: 0.3358 - val_accuracy: 0.8785 - val_loss: 0.3921
Epoch 55/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9090 - loss: 0.2475 - val_accuracy: 0.8902 - val_loss: 0.4336
Epoch 56/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 12s 95ms/step - accuracy: 0.9076 - loss: 0.2406 - val_accuracy: 0.8808 - val_loss: 0.4163
Epoch 57/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9009 - loss: 0.2716 - val_accuracy: 0.8949 - val_loss: 0.3282
Epoch 58/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 93ms/step - accuracy: 0.9062 - loss: 0.2480 - val_accuracy: 0.9019 - val_loss: 0.3606
Epoch 59/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 92ms/step - accuracy: 0.8990 - loss: 0.2681 - val_accuracy: 0.9112 - val_loss: 0.3189
Epoch 60/60
121/121 ━━━━━━━━━━━━━━━━━━━━ 11s 91ms/step - accuracy: 0.9048 - loss: 0.2387 - val_accuracy: 0.8995 - val_loss: 0.3399

Plot Accuracy Graph

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

• As we can see from above results, training and validation is giving better accuracy around 90%, let's evaluate the model on test data

Evaluate model on test data

test_accuracy1 = model1.evaluate(X_test_normalized, y_test_encoded, verbose=1)

15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 18ms/step - accuracy: 0.9021 - loss: 0.3532

• As we can see model 1 is giving around 90% accuray on traning and validation set and also 90% test accuracy.
• Model is generalizing well

Generating predictions on test data

y_pred = model1.predict(X_test_normalized)

15/15 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step

Plot Confusion matrix

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)

# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()

• Considering confusion matrix, Model 1 is giving better predictions for Common Chickweed, Maize, Small-flowered Cranesbill,Scentless Mayweed, Fat hen
• Model has given misclassified predictions on Black grass, seems it confusing Black grass with Loose Silky-bent, also some wrong predictions for Loose Silky-bent as black grass

# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))

                           precision    recall  f1-score   support

              Black-grass       0.64      0.35      0.45        26
                 Charlock       0.84      0.92      0.88        39
                 Cleavers       0.89      0.86      0.88        29
         Common Chickweed       1.00      0.93      0.97        61
             Common wheat       0.83      0.86      0.84        22
                  Fat Hen       0.90      0.96      0.93        48
         Loose Silky-bent       0.80      0.94      0.87        65
                    Maize       0.92      1.00      0.96        22
        Scentless Mayweed       0.91      0.96      0.93        52
          Shepherds Purse       0.83      0.65      0.73        23
Small-flowered Cranesbill       0.96      0.98      0.97        50
               Sugar beet       0.97      0.89      0.93        38

                 accuracy                           0.89       475
                macro avg       0.87      0.86      0.86       475
             weighted avg       0.89      0.89      0.88       475

Comments

• Recall is low for Black-grass and except Shepherds Purse, for other classes recall is above 90%
• Precision is slighly low for Black grass but all other classes have good preicision and f1 score
• Even from confusion matrix, we see model did not perform well for black grass
• Overall accuracy is good, let's check if we can improve further

Model 2 - Model with Reducing Learning Rate & Data Augmentation

Reducing the Learning Rate:

Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.

learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy', patience=3, verbose=1, factor=0.5, min_lr=0.00001)

# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)

# set the rotation_range to 20
train_datagen1 = ImageDataGenerator(rotation_range=20,  # randomly rotate images in the range
        horizontal_flip=True,  # randomly flip images horizontally
        vertical_flip=True  # randomly flip images vertically
    )

# Initializing a sequential model
# Initializing a sequential model
model2 = Sequential()

# Adding first conv layer with 128 filters and Kernel size 3*3, padding 'same'
model2.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(128,128,3)))

# Adding max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2)))

# Add two convolution and max-pooling layers activation = relu
model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))

model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))

model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))

model2.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model2.add(layers.BatchNormalization())
model2.add(MaxPooling2D((2, 2)))

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())

# Adding a fully connected dense layer with 16 neurons
model2.add(Dense(64, activation='relu'))
model2.add(Dropout(0.3))

# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))

# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
model2.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                      │ (None, 126, 126, 32)        │             896 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D)         │ (None, 63, 63, 32)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_1 (Conv2D)                    │ (None, 61, 61, 64)          │          18,496 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization                  │ (None, 61, 61, 64)          │             256 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_1 (MaxPooling2D)       │ (None, 30, 30, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_2 (Conv2D)                    │ (None, 28, 28, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization_1                │ (None, 28, 28, 64)          │             256 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_2 (MaxPooling2D)       │ (None, 14, 14, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_3 (Conv2D)                    │ (None, 12, 12, 64)          │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization_2                │ (None, 12, 12, 64)          │             256 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_3 (MaxPooling2D)       │ (None, 6, 6, 64)            │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_4 (Conv2D)                    │ (None, 4, 4, 64)            │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization_3                │ (None, 4, 4, 64)            │             256 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_4 (MaxPooling2D)       │ (None, 2, 2, 64)            │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 256)                 │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense)                        │ (None, 64)                  │          16,448 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout (Dropout)                    │ (None, 64)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense)                      │ (None, 12)                  │             780 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 148,428 (579.80 KB)

 Trainable params: 147,916 (577.80 KB)

 Non-trainable params: 512 (2.00 KB)

Fitting the model

# fit the model on train data with batch_size=64 and epochs=30
epochs = 50
batch_size = 32

history = model2.fit(train_datagen.flow(X_train_normalized, y_train_encoded, seed=42, shuffle=True),
                                       epochs=epochs,
                                       batch_size=batch_size,
                                       steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                                       validation_data=(X_val_normalized,y_val_encoded),
                                       verbose=1,callbacks=[learning_rate_reduction])

Epoch 1/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 16s 113ms/step - accuracy: 0.2576 - loss: 2.3839 - val_accuracy: 0.0467 - val_loss: 5.1001 - learning_rate: 0.0010
Epoch 2/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.4375 - loss: 1.6614 - val_accuracy: 0.0467 - val_loss: 5.1101 - learning_rate: 0.0010
Epoch 3/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 113ms/step - accuracy: 0.5388 - loss: 1.3733 - val_accuracy: 0.0935 - val_loss: 5.9076 - learning_rate: 0.0010
Epoch 4/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.8170 - val_accuracy: 0.0935 - val_loss: 5.9096 - learning_rate: 0.0010
Epoch 5/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 116ms/step - accuracy: 0.6419 - loss: 1.0307 - val_accuracy: 0.0771 - val_loss: 6.7895 - learning_rate: 0.0010
Epoch 6/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 9s 83ms/step - accuracy: 0.6250 - loss: 1.1705
Epoch 6: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.6250 - loss: 1.1705 - val_accuracy: 0.0771 - val_loss: 6.7641 - learning_rate: 0.0010
Epoch 7/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 112ms/step - accuracy: 0.7215 - loss: 0.8213 - val_accuracy: 0.0794 - val_loss: 6.8274 - learning_rate: 5.0000e-04
Epoch 8/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.5938 - loss: 0.8244 - val_accuracy: 0.0748 - val_loss: 6.9343 - learning_rate: 5.0000e-04
Epoch 9/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.7527 - loss: 0.6844 - val_accuracy: 0.1822 - val_loss: 3.1616 - learning_rate: 5.0000e-04
Epoch 10/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4490 - val_accuracy: 0.1822 - val_loss: 3.2334 - learning_rate: 5.0000e-04
Epoch 11/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.7806 - loss: 0.6340 - val_accuracy: 0.7780 - val_loss: 0.6466 - learning_rate: 5.0000e-04
Epoch 12/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4733 - val_accuracy: 0.7710 - val_loss: 0.6639 - learning_rate: 5.0000e-04
Epoch 13/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8033 - loss: 0.5845 - val_accuracy: 0.7173 - val_loss: 0.8277 - learning_rate: 5.0000e-04
Epoch 14/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 9s 83ms/step - accuracy: 0.7188 - loss: 0.6965
Epoch 14: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7188 - loss: 0.6965 - val_accuracy: 0.7383 - val_loss: 0.7942 - learning_rate: 5.0000e-04
Epoch 15/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8192 - loss: 0.5330 - val_accuracy: 0.8458 - val_loss: 0.4674 - learning_rate: 2.5000e-04
Epoch 16/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4077 - val_accuracy: 0.8458 - val_loss: 0.4868 - learning_rate: 2.5000e-04
Epoch 17/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8275 - loss: 0.5009 - val_accuracy: 0.8388 - val_loss: 0.4613 - learning_rate: 2.5000e-04
Epoch 18/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 91ms/step - accuracy: 0.9062 - loss: 0.3323
Epoch 18: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9062 - loss: 0.3323 - val_accuracy: 0.8435 - val_loss: 0.4621 - learning_rate: 2.5000e-04
Epoch 19/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8344 - loss: 0.4913 - val_accuracy: 0.8201 - val_loss: 0.4692 - learning_rate: 1.2500e-04
Epoch 20/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.7823 - val_accuracy: 0.8248 - val_loss: 0.4618 - learning_rate: 1.2500e-04
Epoch 21/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 107ms/step - accuracy: 0.8245 - loss: 0.4762
Epoch 21: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8246 - loss: 0.4760 - val_accuracy: 0.8201 - val_loss: 0.4610 - learning_rate: 1.2500e-04
Epoch 22/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7500 - loss: 0.5539 - val_accuracy: 0.8224 - val_loss: 0.4585 - learning_rate: 6.2500e-05
Epoch 23/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8552 - loss: 0.4351 - val_accuracy: 0.8388 - val_loss: 0.4810 - learning_rate: 6.2500e-05
Epoch 24/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 86ms/step - accuracy: 0.8125 - loss: 0.4305
Epoch 24: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4305 - val_accuracy: 0.8388 - val_loss: 0.4810 - learning_rate: 6.2500e-05
Epoch 25/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8505 - loss: 0.4267 - val_accuracy: 0.8528 - val_loss: 0.4049 - learning_rate: 3.1250e-05
Epoch 26/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.5223 - val_accuracy: 0.8528 - val_loss: 0.4045 - learning_rate: 3.1250e-05
Epoch 27/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8607 - loss: 0.4184 - val_accuracy: 0.8505 - val_loss: 0.4098 - learning_rate: 3.1250e-05
Epoch 28/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 85ms/step - accuracy: 0.8438 - loss: 0.4249
Epoch 28: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.4249 - val_accuracy: 0.8505 - val_loss: 0.4103 - learning_rate: 3.1250e-05
Epoch 29/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8486 - loss: 0.4248 - val_accuracy: 0.8575 - val_loss: 0.3996 - learning_rate: 1.5625e-05
Epoch 30/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4711 - val_accuracy: 0.8575 - val_loss: 0.3993 - learning_rate: 1.5625e-05
Epoch 31/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8475 - loss: 0.4188 - val_accuracy: 0.8668 - val_loss: 0.3914 - learning_rate: 1.5625e-05
Epoch 32/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8125 - loss: 0.4689 - val_accuracy: 0.8668 - val_loss: 0.3911 - learning_rate: 1.5625e-05
Epoch 33/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8483 - loss: 0.4209 - val_accuracy: 0.8621 - val_loss: 0.3941 - learning_rate: 1.5625e-05
Epoch 34/50
  1/120 ━━━━━━━━━━━━━━━━━━━━ 10s 92ms/step - accuracy: 0.8438 - loss: 0.3642
Epoch 34: ReduceLROnPlateau reducing learning rate to 1e-05.
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.3642 - val_accuracy: 0.8621 - val_loss: 0.3935 - learning_rate: 1.5625e-05
Epoch 35/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8495 - loss: 0.4235 - val_accuracy: 0.8621 - val_loss: 0.3854 - learning_rate: 1.0000e-05
Epoch 36/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9375 - loss: 0.2938 - val_accuracy: 0.8621 - val_loss: 0.3859 - learning_rate: 1.0000e-05
Epoch 37/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8459 - loss: 0.4025 - val_accuracy: 0.8621 - val_loss: 0.3887 - learning_rate: 1.0000e-05
Epoch 38/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9688 - loss: 0.1799 - val_accuracy: 0.8598 - val_loss: 0.3887 - learning_rate: 1.0000e-05
Epoch 39/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8495 - loss: 0.3958 - val_accuracy: 0.8551 - val_loss: 0.3954 - learning_rate: 1.0000e-05
Epoch 40/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9375 - loss: 0.2187 - val_accuracy: 0.8551 - val_loss: 0.3956 - learning_rate: 1.0000e-05
Epoch 41/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 111ms/step - accuracy: 0.8516 - loss: 0.4143 - val_accuracy: 0.8621 - val_loss: 0.3881 - learning_rate: 1.0000e-05
Epoch 42/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8438 - loss: 0.3724 - val_accuracy: 0.8621 - val_loss: 0.3881 - learning_rate: 1.0000e-05
Epoch 43/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8503 - loss: 0.4286 - val_accuracy: 0.8621 - val_loss: 0.3913 - learning_rate: 1.0000e-05
Epoch 44/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7188 - loss: 0.7387 - val_accuracy: 0.8621 - val_loss: 0.3917 - learning_rate: 1.0000e-05
Epoch 45/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 14s 114ms/step - accuracy: 0.8566 - loss: 0.3949 - val_accuracy: 0.8575 - val_loss: 0.3864 - learning_rate: 1.0000e-05
Epoch 46/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8750 - loss: 0.3060 - val_accuracy: 0.8575 - val_loss: 0.3869 - learning_rate: 1.0000e-05
Epoch 47/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 110ms/step - accuracy: 0.8632 - loss: 0.3852 - val_accuracy: 0.8598 - val_loss: 0.3980 - learning_rate: 1.0000e-05
Epoch 48/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7812 - loss: 0.7183 - val_accuracy: 0.8598 - val_loss: 0.3982 - learning_rate: 1.0000e-05
Epoch 49/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 13s 109ms/step - accuracy: 0.8502 - loss: 0.4178 - val_accuracy: 0.8598 - val_loss: 0.3914 - learning_rate: 1.0000e-05
Epoch 50/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.9688 - loss: 0.1456 - val_accuracy: 0.8598 - val_loss: 0.3912 - learning_rate: 1.0000e-05

Plot Accuracy Graph

plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

• As we can see the accuracy has been improved with data augmentation compare to earlier model

Evaluate the model on test data

accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)

15/15 - 0s - 22ms/step - accuracy: 0.8505 - loss: 0.4334

Generating the predictions for test data

# Here we would get the output as probablities for each category
y_pred=model2.predict(X_test_normalized)

15/15 ━━━━━━━━━━━━━━━━━━━━ 1s 29ms/step

# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)

# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Confusion Matrix');
ax.xaxis.set_ticklabels(list(lb.classes_),rotation=90)
ax.yaxis.set_ticklabels(list(lb.classes_),rotation=0)
plt.show()

# Plotting Classification Report
from sklearn.metrics import classification_report
print(classification_report(y_test_arg, y_pred_arg, target_names=list(lb.classes_)))

                           precision    recall  f1-score   support

              Black-grass       0.33      0.19      0.24        26
                 Charlock       0.86      0.95      0.90        39
                 Cleavers       0.89      0.83      0.86        29
         Common Chickweed       0.93      0.93      0.93        61
             Common wheat       0.79      0.50      0.61        22
                  Fat Hen       0.94      0.92      0.93        48
         Loose Silky-bent       0.71      0.92      0.81        65
                    Maize       0.81      1.00      0.90        22
        Scentless Mayweed       0.85      0.96      0.90        52
          Shepherds Purse       1.00      0.57      0.72        23
Small-flowered Cranesbill       0.94      0.92      0.93        50
               Sugar beet       0.97      0.92      0.95        38

                 accuracy                           0.85       475
                macro avg       0.84      0.80      0.81       475
             weighted avg       0.85      0.85      0.84       475

• The test accuracy of the model is around 85% which is less than earlier model
• Recall of brack grass is also reduced so
• Overall model performance is less than earlier model

Model 3 - Using VGG16 architecture

from tensorflow.keras.models import Model
from keras.applications.vgg16 import VGG16

vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (128,128,3))
vgg_model.summary()

Model: "vgg16"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ input_layer_3 (InputLayer)           │ (None, 128, 128, 3)         │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block1_conv1 (Conv2D)                │ (None, 128, 128, 64)        │           1,792 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block1_conv2 (Conv2D)                │ (None, 128, 128, 64)        │          36,928 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block1_pool (MaxPooling2D)           │ (None, 64, 64, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block2_conv1 (Conv2D)                │ (None, 64, 64, 128)         │          73,856 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block2_conv2 (Conv2D)                │ (None, 64, 64, 128)         │         147,584 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block2_pool (MaxPooling2D)           │ (None, 32, 32, 128)         │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block3_conv1 (Conv2D)                │ (None, 32, 32, 256)         │         295,168 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block3_conv2 (Conv2D)                │ (None, 32, 32, 256)         │         590,080 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block3_conv3 (Conv2D)                │ (None, 32, 32, 256)         │         590,080 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block3_pool (MaxPooling2D)           │ (None, 16, 16, 256)         │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block4_conv1 (Conv2D)                │ (None, 16, 16, 512)         │       1,180,160 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block4_conv2 (Conv2D)                │ (None, 16, 16, 512)         │       2,359,808 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block4_conv3 (Conv2D)                │ (None, 16, 16, 512)         │       2,359,808 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block4_pool (MaxPooling2D)           │ (None, 8, 8, 512)           │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block5_conv1 (Conv2D)                │ (None, 8, 8, 512)           │       2,359,808 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block5_conv2 (Conv2D)                │ (None, 8, 8, 512)           │       2,359,808 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block5_conv3 (Conv2D)                │ (None, 8, 8, 512)           │       2,359,808 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ block5_pool (MaxPooling2D)           │ (None, 4, 4, 512)           │               0 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 14,714,688 (56.13 MB)

 Trainable params: 14,714,688 (56.13 MB)

 Non-trainable params: 0 (0.00 B)

# Making all the layers of the VGG model non-trainable. i.e. freezing them
for layer in vgg_model.layers:
    layer.trainable = False

new_model = Sequential()

# Adding the convolutional part of the VGG16 model from above
new_model.add(vgg_model)

# Flattening the output of the VGG16 model because it is from a convolutional layer
new_model.add(Flatten())

# Adding a dense output layer
new_model.add(layers.BatchNormalization())
new_model.add(Dense(32, activation='relu'))
new_model.add(Dropout(0.2))

new_model.add(layers.BatchNormalization())
new_model.add(Dense(16, activation='relu'))

new_model.add(Dense(12, activation='softmax'))
opt=Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999)
# Compile model
new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

# Generating the summary of the model
new_model.summary()

Model: "sequential_2"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                         ┃ Output Shape                ┃         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ vgg16 (Functional)                   │ (None, 4, 4, 512)           │      14,714,688 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten_2 (Flatten)                  │ (None, 8192)                │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization_6                │ (None, 8192)                │          32,768 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_5 (Dense)                      │ (None, 32)                  │         262,176 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout_2 (Dropout)                  │ (None, 32)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization_7                │ (None, 32)                  │             128 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_6 (Dense)                      │ (None, 16)                  │             528 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_7 (Dense)                      │ (None, 12)                  │             204 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘

 Total params: 15,010,492 (57.26 MB)

 Trainable params: 279,356 (1.07 MB)

 Non-trainable params: 14,731,136 (56.19 MB)

callback_es = tf.keras.callbacks.EarlyStopping(monitor='val_accuracy', patience=10, min_delta=0.0001, restore_best_weights=True)

# Epochs
epochs = 50
# Batch size
batch_size = 32

history_vgg16 = new_model.fit(train_datagen1.flow(X_train_normalized, y_train_encoded,
                                       seed=42,
                                       shuffle=True),
                    epochs=epochs,
                    batch_size=batch_size,         
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),
                    verbose=1,callbacks=[callback_es])

Epoch 1/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 69s 563ms/step - accuracy: 0.2206 - loss: 2.2362 - val_accuracy: 0.4813 - val_loss: 1.9825
Epoch 2/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.4375 - loss: 1.6743 - val_accuracy: 0.4836 - val_loss: 1.9779
Epoch 3/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 69s 577ms/step - accuracy: 0.4755 - loss: 1.6552 - val_accuracy: 0.6332 - val_loss: 1.5963
Epoch 4/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 7s 58ms/step - accuracy: 0.5938 - loss: 1.3362 - val_accuracy: 0.6355 - val_loss: 1.5940
Epoch 5/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.5722 - loss: 1.4546 - val_accuracy: 0.6682 - val_loss: 1.3327
Epoch 6/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.5312 - loss: 1.4326 - val_accuracy: 0.6706 - val_loss: 1.3305
Epoch 7/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.6226 - loss: 1.3176 - val_accuracy: 0.7033 - val_loss: 1.1366
Epoch 8/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8438 - loss: 1.0012 - val_accuracy: 0.6986 - val_loss: 1.1357
Epoch 9/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 585ms/step - accuracy: 0.6634 - loss: 1.2184 - val_accuracy: 0.7313 - val_loss: 1.0107
Epoch 10/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7188 - loss: 1.0065 - val_accuracy: 0.7313 - val_loss: 1.0094
Epoch 11/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.6904 - loss: 1.1052 - val_accuracy: 0.7523 - val_loss: 0.9257
Epoch 12/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7188 - loss: 1.0463 - val_accuracy: 0.7523 - val_loss: 0.9249
Epoch 13/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.7013 - loss: 1.0559 - val_accuracy: 0.7710 - val_loss: 0.8456
Epoch 14/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.6250 - loss: 1.1622 - val_accuracy: 0.7710 - val_loss: 0.8449
Epoch 15/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.7384 - loss: 0.9733 - val_accuracy: 0.7757 - val_loss: 0.8102
Epoch 16/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.6562 - loss: 0.9414 - val_accuracy: 0.7757 - val_loss: 0.8095
Epoch 17/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7520 - loss: 0.9019 - val_accuracy: 0.7967 - val_loss: 0.7381
Epoch 18/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8125 - loss: 0.7886 - val_accuracy: 0.7967 - val_loss: 0.7377
Epoch 19/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.7526 - loss: 0.8858 - val_accuracy: 0.8061 - val_loss: 0.6941
Epoch 20/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7500 - loss: 0.9106 - val_accuracy: 0.8061 - val_loss: 0.6937
Epoch 21/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7927 - loss: 0.7812 - val_accuracy: 0.8201 - val_loss: 0.6541
Epoch 22/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.7812 - loss: 0.7311 - val_accuracy: 0.8201 - val_loss: 0.6537
Epoch 23/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7908 - loss: 0.7521 - val_accuracy: 0.8248 - val_loss: 0.6254
Epoch 24/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.7500 - loss: 0.7910 - val_accuracy: 0.8271 - val_loss: 0.6250
Epoch 25/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 584ms/step - accuracy: 0.7975 - loss: 0.7030 - val_accuracy: 0.8224 - val_loss: 0.6059
Epoch 26/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 7s 59ms/step - accuracy: 0.6875 - loss: 0.9090 - val_accuracy: 0.8248 - val_loss: 0.6057
Epoch 27/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.7976 - loss: 0.7051 - val_accuracy: 0.8388 - val_loss: 0.5781
Epoch 28/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.9375 - loss: 0.3988 - val_accuracy: 0.8364 - val_loss: 0.5782
Epoch 29/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8076 - loss: 0.6738 - val_accuracy: 0.8435 - val_loss: 0.5445
Epoch 30/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.9688 - loss: 0.3688 - val_accuracy: 0.8458 - val_loss: 0.5448
Epoch 31/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 587ms/step - accuracy: 0.8258 - loss: 0.6305 - val_accuracy: 0.8481 - val_loss: 0.5390
Epoch 32/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.6562 - loss: 0.8649 - val_accuracy: 0.8505 - val_loss: 0.5381
Epoch 33/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 586ms/step - accuracy: 0.8321 - loss: 0.5788 - val_accuracy: 0.8411 - val_loss: 0.5211
Epoch 34/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8438 - loss: 0.4275 - val_accuracy: 0.8411 - val_loss: 0.5206
Epoch 35/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 588ms/step - accuracy: 0.8394 - loss: 0.5793 - val_accuracy: 0.8481 - val_loss: 0.5161
Epoch 36/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7500 - loss: 0.6967 - val_accuracy: 0.8481 - val_loss: 0.5159
Epoch 37/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 586ms/step - accuracy: 0.8258 - loss: 0.5699 - val_accuracy: 0.8505 - val_loss: 0.4921
Epoch 38/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.7812 - loss: 0.5682 - val_accuracy: 0.8505 - val_loss: 0.4919
Epoch 39/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 587ms/step - accuracy: 0.8394 - loss: 0.5317 - val_accuracy: 0.8598 - val_loss: 0.4757
Epoch 40/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8438 - loss: 0.5973 - val_accuracy: 0.8598 - val_loss: 0.4750
Epoch 41/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8367 - loss: 0.5417 - val_accuracy: 0.8575 - val_loss: 0.4593
Epoch 42/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.9062 - loss: 0.4695 - val_accuracy: 0.8551 - val_loss: 0.4585
Epoch 43/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 585ms/step - accuracy: 0.8696 - loss: 0.4777 - val_accuracy: 0.8598 - val_loss: 0.4504
Epoch 44/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 60ms/step - accuracy: 0.8125 - loss: 0.4858 - val_accuracy: 0.8598 - val_loss: 0.4505
Epoch 45/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 72s 598ms/step - accuracy: 0.8548 - loss: 0.4925 - val_accuracy: 0.8575 - val_loss: 0.4366
Epoch 46/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.8125 - loss: 0.5229 - val_accuracy: 0.8528 - val_loss: 0.4363
Epoch 47/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 582ms/step - accuracy: 0.8653 - loss: 0.4448 - val_accuracy: 0.8458 - val_loss: 0.4315
Epoch 48/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 8s 59ms/step - accuracy: 0.9688 - loss: 0.3226 - val_accuracy: 0.8435 - val_loss: 0.4326
Epoch 49/50
120/120 ━━━━━━━━━━━━━━━━━━━━ 70s 583ms/step - accuracy: 0.8486 - loss: 0.4846 - val_accuracy: 0.8505 - val_loss: 0.4192

plt.plot(history_vgg16.history['accuracy'])
plt.plot(history_vgg16.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=1)

15/15 ━━━━━━━━━━━━━━━━━━━━ 7s 459ms/step - accuracy: 0.8599 - loss: 0.4553

As VGG16 models does not perform better than earlier model, let's skip the furthern evalution of model on test data

Final Model

pd.DataFrame({'Models':['Model 0 - Base Model with CNN layers','Model 1 - Data Augmentation and Callback ','Model 2 -Data Augmentation and Reducing learning rate', 'Model 3 - Transfer learning with VGG16 architecture'],'Train Accuracy':['89%','91%','96%','85%'],'Validation Accuracy':['89%','90%','86%','85%'],'Test Accuracy':['88%','90%','85%','86%']})

	Models	Train Accuracy	Validation Accuracy	Test Accuracy
0	Model 0 - Base Model with CNN layers	89%	89%	88%
1	Model 1 - Data Augmentation and Callback	91%	90%	90%
2	Model 2 -Data Augmentation and Reducing learni...	96%	86%	85%
3	Model 3 - Transfer learning with VGG16 archite...	85%	85%	86%

• Considering the accuracy of all above models. 'Model 1 - Data Augementation and Callback' gives better accuracy score for train, validation and test sets i.e. around 90% accuracy, so will go ahead with Model 1 as best model.
• Let's visualize and predict the test data samples using Model 1

Visualizing the prediction

# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[2].reshape(1,128,128,3)))))   # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[2])                                               # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[33].reshape(1,128,128,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[33])                                              # using inverse_transform() to get the output label from the output vector

plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
print('Predicted Label', lb.inverse_transform(model1.predict((X_test_normalized[36].reshape(1,128,128,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', lb.inverse_transform(y_test_encoded)[36]) 

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step
Predicted Label ['Small-flowered Cranesbill']
True Label Small-flowered Cranesbill

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 51ms/step
Predicted Label ['Cleavers']
True Label Cleavers

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step
Predicted Label ['Shepherds Purse']
True Label Shepherds Purse

Conclusion:

• As we can see from predictions model has predicted all 3 random classes correctly. With some changes the parameters model may perform better.

• We have built a CNN-model to predict the class of a plant, which works quite well. (Increasing number of epochs and/or adding layers to a model can even increase the performance)

• CNN with Batch Normalization, Maxpooling, dropouts + Dense layers is a good combination for image classification

Actionable Insights and Business Recommendations

Actionable Insights

• Analyze classification accuracy and confusion matrix to identify misclassified seedling classes.
• Models can be further improved by training on different parameters and find better optimized combination of parameters
• Assess dataset balance and image quality to address potential biases and improve model training.
• Evaluate prediction speed for suitability in real-time applications.
• Incorporate active learning and retraining with new data to enhance model robustness.
• With better computational resources models can be further trained and improved on better image quality or larger sizes.
• Different transfer learning achitectures can be used and train to check for better results
• Model can be further improved by different sets of data augementation sets/combinations

Business Recommendations

• Companies can gather more data for various variety of seedlings to improve model performance.
• Precision Agriculture: Use the model for automated weed detection, crop monitoring, and early disease identification.
• Cost Optimization: Automate labor-intensive tasks like seedling identification to save costs and resources.
• Product Development: Create a mobile app for instant seedling classification and care advice.
• Revenue Streams: Offer subscription services for advanced features and license anonymized data for research.
• Partnerships: Collaborate with agricultural bodies and seed companies to tailor and scale the solution.
• Custom Solutions for Seed Companies:Partner with seed companies to customize the CNN model for their proprietary seed varieties, offering them a competitive edge.

---